EdgeSync-LLM, a new KV cache fragment engine for on-device LLM inference, stores and retrieves transformer KV tensors via HNSW approximate nearest-neighbor search, enabling exact hits at ~8ms TTFT and…
QVAC, an open-source ecosystem for building local-first, peer-to-peer AI applications, has been released. It enables running AI tasks like LLMs, speech, and RAG locally across platforms or via P2P inf…
Developer jasonmoo released Jargo, a Go port of the Pipecat conversational-AI framework, enabling real-time voice agents with WebRTC audio, streaming transcription-to-speech pipelines, and turn-taking…
AMD engineer Steven Xiao contributed an ONNX Runtime back-end for FFmpeg's DNN filter, enabling AI model inferencing on multiple GPU and NPU platforms including NVIDIA CUDA, Windows DirectML, and AMD …
Neonmem 0.9.7 introduces a two-level importer that separates folders and files into a searchable knowledge pool and agent chats into typed memories, using IBM Granite-30M embeddings via ONNX Runtime f…
Baidu released PP-OCRv6, a family of OCR models scaling from 1.5M to 34.5M parameters, supporting 50 languages. The medium model achieves 86.2% detection Hmean and 83.2% recognition accuracy, improvin…
HuggingFace released Transformers.js v4, a major update featuring a new WebGPU backend rewritten in C++ for faster AI model inference in browsers, Node, Bun, and Deno. The release adds support for lar…
Neo launched an MCP server that lets users offload AI tasks from Claude, reducing costs by 62% and speeding up runtime by 37% in benchmarks. The tool integrates with Claude Code and other MCP clients,…
The x86 Ecosystem Advisory Group released the AI Compute Extensions (ACE) Specification on June 15, 2026, defining standardized x86 extensions for matrix multiplication and tile registers to accelerat…
A developer built an offline skin lesion segmenter for mobile devices by optimizing the Med-SAM model with CoreML and ONNX Runtime. The approach enables real-time, privacy-preserving segmentation on i…
VecTrade.io engineers detail how to build real-time machine learning inference pipelines for algorithmic trading, using in-memory sliding ring buffers for feature extraction and multiprocessing worker…
Mozilla has integrated an on-device ML inference runtime into Firefox, using ONNX Runtime and Transformers.js, to power features like PDF.js alt text generation and Smart Tab grouping. The runtime run…
Outgate AI released og-local, a local privacy proxy that intercepts API calls from coding agents to detect and redact personally identifiable information and secrets before they reach third-party larg…
The W3C draft specification WebNN defines a graph-based neural network API that enables JavaScript to submit static model graphs to native OS runtimes, according to a November 21, 2025 blog post on zi…
A developer successfully offloaded automatic speech recognition (ASR) for a Home Assistant voice pipeline from CPU to the Intel NPU on a Core Ultra 7 265K processor, achieving 3-6× faster inference an…